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Abstract 

We prove that the relation of bisimilarity between countable labelled transition 
systems is not Borel, by reducing bounded classes of countable wellorders continu- 
ously to it. 

This has an impact on the theory of probabilistic and nondeterministic processes 
over uncountable spaces, since the proofs of logical characterizations of bisimilarity 
based on the unique structure theorem for analytic spaces require a countable logic 
whose formulas have measurable semantics. Our reduction shows that such a logic 
does not exist in the case of image- infinite process. 

1 Introduction 

Markov decision processes over continuous state-space are an appropriate framework to 
study and formalize systems that involve continuously valued variables such as those 
arising in physics, biology, and economics; and where some of those variables are known 
only in a probabilistic way. 

In this direction, labelled Markov processes (LMP) where developed in j^, ^ by De- 
sharnais et alter. An LMP has a labelled set of actions that encode interaction with 
the environment; thus LMP are a reactive model in which there are different transi- 
tion subprobabilities for each action. In this model, uncertainty is (only) considered 
to be probabilistic; therefore, LMP can be regarded as generalization of deterministic 
processes. 

Models that include both probabilism and internal nondeterminism arise naturally, 
e.g., by abstraction of LMP. In the discrete case, probabilistic automata are an example. 
Over uncountable state spaces, the common generalization of LMP and probabilistic 
automata are nondeterministic labelled Markov processes (NLMP) NLMP allow, 

for each state and each action, a (possibly infinite) set of probabilistic behaviors. A 
different approach, using super-additive functions, is proposed in Q. 

In 0, 0] the problem of defining appropriate notions of bisimulation and finding logical 
characterizations for NLMP was addressed. It turns out that there are three different 
notions of bisimilarity: traditional, state-based and event-based. Two of these notions 
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collapse to the state bisimilarity when one drops non-determinism (i.e., for LMP), and 
the third one is analogous to the concept for LMP bearing the same name. Furthermore, 
when the state-space is analytic, state-based and event-based bisimilarity coincide for 
LMP [2]. This was proved alternatively, over Polish spaces, in Doberkat [8(. 

In both LMP and NLMP, event bisimilarity is a relation naturally characterized by 
a modal logic, in the sense that two states satisfy exactly the same formulas if and only 
if they are bisimilar eventwise. When event and traditional bisimilarities coincides, we 
speak of a logical characterization of bisimilarity, emphasizing the role of traditional 
(state) bisimilarity as a the most natural generalization of probabilistic bisimilarity by 
Larsen and Skou |Hj to NLMP (LMP). 

The unique structure theorem provides a very clear way to prove the logical charac- 
terization of bisimilarity for LMP with an analytic state-space, as in the work of Danos 
et al. [2]. The argument there can be generalized to encompass image- finite NLMP (i.e., 
having a finite number of probabilistic behaviors for each pair (state, action)) over an- 
alytic state spaces. Actually, a proof strategy can be found in 0, every countable 
'measurable' logic Jzf satisfying certain local restrictions must characterize bisimilarity. 
(Here we call a logic measurable if the extension of each formula is a measurable subset 
of the state space.) Both countability and measurability requirements are necessary for 
the proof to work. 

In this paper we show that the relation of bisimilarity is not Borel in an appropriate 
Polish space, and therefore we prove that there is no countable measurable logic that 
characterizes it. In the next section we review some of the known results on NLMP, 
describing the available notions of bisimulation. Most calculations in the paper will be 
carried on a simpler model also described in Section 2. In Section 3 we use some machinery 
of sequence spaces and unwinding of labelled transition systems to assess the complexity 
of the relation of bisimilarity. The final section contains some concluding remarks. 



2 Review of NLMP 
2.1 Basic definitions 

All the material of this section appears in Qj. Let (S, £) be a measurable space. The set 
A(S) of probability measures on (S, E) has a natural cr-algebra A(E) = a({A- q (Q) : q G 
Q, Q G £}), where A^(Q) = {/iG A(S) : fi(Q) > q}. This is the least cr-algebra making 
evaluation \i fi(Q) measurable. 

Recall that a Markov kernel on (S, E) is a measurable map T : (S, E) — > (A(S), A(E)). 
The following definitions generalize this concept by enlarging the codomain to the family 
of all measurable sets of probability measures (A(E)), and construct a cr-algebra for this 
family in order to be able to say that T is measurable. 

Definition 1. if(A(£)) is the least cr-algebra containing all sets = G A(E) : 

Cnc/0} withe eA(E). 

Definition 2. A nondeterministic labeled Markov process (NLMP) is a tuple (S, E, {T a : 
a G L}) where E is a cr-algebra on the set of states S, and for each label a G L, 
T a : S — > A(E) is measurable. 
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The motivation for the previous definitions is that we want the event "there exists a 
probabilistic behavior from s such that ..." to be measurable; to be able to calculate 
the probability of such event (cf. the semantics of the logic below). 

Some notation concerning binary relations will be needed to define bisimulations. Let 
R a binary relation over S. A set Q is R-closed if Q 3 x R y implies y G Q. E(i2) is 
the cr-algebra of i?-closed sets in E. If //, // are measures defined on E, we write \x R \Jt! 
if they coincide in E(i2). Lastly, let S be a subset of Pow(S'), the powerset of S. The 
relation 1Z (H) is given by: 

(s,t)eK(Z) VQ G H : s gQ &t eQ. 

Definition 3. 1. An event bisimulation on a NLMP (S,H,{T a : a G L}) is a sub-cx- 
algebra A of E such that T a : (S,A) — > (A(E), H(A(A))) is measurable for each 
a G L. We also say that a relation i? is an event bisimulation if there is an event 
bisimulation H such that i? = 7^.(2). 

2. A relation i? C S x 5 is a state bisimulation if it is symmetric and for all a G L, 
s R t implies V£ G A(E(i?)) : T a (s) 0^0 T a (t) n f ^ 0. 

3. A relation i? is a traditional bisimulation if it is symmetric and for all a G L, s i? t 
implies that for all /i G T a (s) there exists // G T a (t) such that fi R fi'. 

We say that s,t <E S are traditionally (resp. state-, event-) bisimilar, denoted by s ~ t t 
(s ~ s t, s ~ e t), if there is a traditional (state, event) bisimulation R such that s Rt. 

We want to stress the fact that each notion of bisimulation/bisimilarity is defined 
relative to a particular NLMP. Event bisimulation is a straightforward generalization 
of the same concept for LMP, and it is the one that is more "compatible" with the 
measurable structure of the base space. For LMP, it is equivalent to the existence of 
a cospan of morphisms; see (ij. Traditional bisimulation is in a sense the most faithful 
generalization of both probabilistic bisimulation by Larsen and Skou and the standard 
notion of bisimulation for non deterministic processes, e.g., LTS. Finally, state bisimilarity 
is a good trade-off between the other two, since it is generally finer than event bisimilarity 
but it is a little more respectful to the measurable structure than the traditional version 

0. 

Finally, we introduce the logic ££ which is given by the productions below. There are 
two kinds of formulas: one that is interpreted on states, and another that is interpreted 
on measures. 

Lp = T | <fi A y? 2 | (a)ip 
i> = Vie/V'i I ^ I [<p]> 9 

where a G L, I is a countable index set, and q G Q D [0, 1]. We denote by Jzf the set of 
all formulas generated by the first production. Given a NLMP (S,H,T), the semantics 
is the following: 

Ul = s IV^] = UM 

a <p 2 j = m n M H>] =s\M 

l(a)^}=T-\H m ) IM>J = a^(M) 
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where [x] denotes the extension of \- ft can be proved by induction that all the sets [x] 
are measurable in the respective spaces. Let [Jzf] = \ ■ <p G «5f}. 

Theorem 4. T/ie logic Jz? completely characterizes event bisimulation. In other words, 
ft([JSfl) = ~ e . 

Theorem 5. ~ t C ~ s C ~ e = ft([JS?]). 

Recall that a topological space is Polish if it is separable and completely metrizable 
and an analytic space is the image of a continuous map between Polish spaces. The 
following Lemma summarizes the strategy to prove completeness of a logic for traditional 
bisimilarity. 

Lemma 6. Let (S, £,T) be a NLMP with (S, E) being an analytic Borel space. Let £ 
be a logic s.t. (i) £ contains operators T and A with the usual semantics; (ii) for every 
formula G £, [y?J G S; fmj £/ie sei of all formulas in £ is countable; and (iv) for every 
s 7£(£) t one? every G T a (s) there exists // G T a (t) such that Wip G £, ([<£>]) = /AM)- 
Then, two logically equivalent states s, t are traditionally bisimilar. 

The proof of this lemma is based on the unique structure theorem for analytic Borel 
spaces (S, £): every countably generated sub-cr-algebra of £ that separates points must 
already be E. 

By using this Lemma we were able to prove that a countable fragment of ££ was 
complete for traditional bisimilarity over image-finite NLMP. The next step would be to 
prove a similar result for image-countable processes, and the safest way to test this is in 
a more "discrete" setting. In the next section we consider a restricted class of processes. 



2.2 Measurable LTS 

Many interesting (counter)examples can be constructed by considering non-probabilistic 
NLMP, i.e., one S = (S, E, {T a : a G L}) such that for all a G L and s G S, T a (s) consists 
entirely of point-masses (i.e., Dirac measures). In this section, we will giv e a slightly 



simpler presentation of this kind of processes, that appears in Wolovick [14] . Actually 



these processes are essentially a labelled transition system (LTS) over a measurable space. 

Definition 7. A measurable labelled transition system (MLTS) is a tuple S = (S, E, {T a : 
a G L}) such that (S, E) is a measurable space and for each label a G L, T a : (S, E) — y 
(£,if(E)) is a measurable map. 

If we write (a)Q (0<5 m case L is a singleton) for {s : T a (s) fl Q ^ 0}, then the 
previous requirement on T a amounts to asking E to be stable under the map (a): for 
all Q G E, (a)Q G E. This observation can be generalized to event bisimulations. The 
following lemma appears in [4|. 

Lemma 8. 1. A a-algebra A C E is an event bisimulation on S if and only if it is 
stable under the mapping (a). 

2. A symmetric relation R is a state bisimulation on S if and only if for all s,t G S 
such that s Rt, it holds that for all Q G £(/?), s G (a)Q <^>i G (a)Q. 
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3. A symmetric relation R is a traditional bisimulation on S if and only if for all 
s,t £ S and u £ T a (s), if s Rt then there exists v £ T a (t) such that u 1Z(H(R)) v. 

Since for every relation one has R C TZ(H(R)), standard bisimilarity for LTS is 
also a traditional bisimulation of MLTS. Observe also that the logic Jzf is the same as 
Hennessy-Milner logic with countable conjunctions (and disjunctions) on this family of 
processes, and hence it characterizes standard bisimulation for image-countable MLTS. 
By appealing to Theorem |5] we can state 

Proposition 9. For image-countable MLTS, all kinds of bisimilarities (traditional, state, 
event and standard) coincide. 

We shall henceforth drop the subindexes and use simply ~. 

Let's return to the problem left open at the end of the previous section. To apply 
Lemma El the candidate logic should satisfy several requirements. So the first question 
is if there actually exists any countable logic that characterizes bisimilarity for countable 
LTS. The answer is given by the following 

Example 10 (X. Caicedo). There are at most 2 K ° (bisimilarity classes of) countable LTS 
over N. Hence there is an injective function / from bisimilarity classes to Pow(N). Our 
'logic' will consist of countably many atomic formulas P n (n £ N) with the following 
semantics: 

S,s|=P n <=> n£/([(S, S )y, 

where [-]^ denotes ~-classes of equivalence. The logic ££x '■= {Pn '■ n £ N} is sound and 
complete for bisimilarity. 

The logic J*5c is devised in a non-constructive manner, and the main result in this 
work is to show that actually the extensions of formulas of such a countable logic cannot 
be Borel sets, confirming the intuition that formulas in J?x cannot be conceived as any 
reasonable kind of "test" on a process. 

3 The main result 

We will use some concepts from sequence (zero-dimensional) spaces. The set of all finite 
sequences of natural numbers will be denoted by N*. The empty sequence will be denoted 
by e. The ith element of a sequence s £ N* will be denoted by s l ; hence s = (s°, . . . , s' s ' _1 ), 
where \s\ is the length of s. The sequence s^n, where s £ N* and n £ N is the result of 
appending n at the end of s: s^n = (s°, . . . , s' s ' _1 , n). 

Let A be countable, and consider the discrete topology on it. The product space A N of 
all infinite sequences of elements of A is Polish and has a (clopen) basis given by the sets 
Cf = {x £ A N : f C x}, where / a finite function. When A = 2, we obtain the Cantor 
space Pow(N). In general, for every countable set B, we regard Pow(B) as a compact 
Hausdorff space with basic open sets {XCB: PCXLNnX = 0}, where P,N C B 
are finite. In case B = Nwe obtain exactly the basis given by {Cf}. 

A binary structure (N, R) can be represented by a point in Rel = 2 NxN . The set LO 
of strict linear orders is closed in Rel and hence a Polish space. A tree on N is a subset 
of N* closed by taking prefixes. The set Trpj of all trees on N is a closed subset in 2 N . 
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We will also use the standard notation for classes of pointsets arising in descriptive 
set theory; namely, £?(X) (II? (X)) are the open (closed) sets of a topological space 
X, £°(X) (n°(X)) are countable unions (intersections) of sets in IlJ(X) (S^(X)) with 
< a. Then the family of Borel subsets of X is B(X) = |L <(1>1 S° (X) = {J Q<u)i 11° (X). 
S}(X) denotes the family of analytic subsets of X (see [lOy ) . All these classes are pre- 
served by taking continuous preimages and finite unions and intersections. We will usually 
omit the reference to the space X. 

3.1 Bisimilarity is not Borel 

We first recall a standard construction of trees from linear orders. Given a strict linear 
order (M, R) we may define a new countable structure (Tr, <) with < G 2 N * xN * as follows: 

• T R = {s G N* : sH- 1 R s^- 2 R . . . R s°} U {e}. 

• s < s' -<=>- 3n G N : s^n = s'. 

The tree T R consists of all finite decreasing sequences in (N, R). 
Lemma 11. The map R^Tr from LO to 2 N * is continuous. 

Proof. We will show that the function from 2 NxN to 2 N * with the same definition is 
continuous. Assume S\, . . . , s n , t\, . . . , t m G N*. Then 

Vz < n, Vj : < j < | Sj | ^> s{ R s| _1 
Vi < m, 3j : < j < \U\ & 4 R t{~\ 

The last condition involve finite intersections and unions of clopen sets, and hence it is 
open. For a little more detail, let P = {s l5 . . . , s„}, N = {ti, . . . , t m }. Hence 

(T.)- l ({T CN*:PCT&XHT = 0}) = 

= {i?CNxN:£/ Cfl}n P| (J {R C N X N : D R = 0}, 

i<m 0<j"<|ti| 

where U = {(s{, sf 1 ) : % < n, < j < \ Si \} and V i;j = {{tl.t^ 1 )}. □ 

The main idea is that we can distinguish wellorders (N, R) just by checking the bisimi- 
larity type of (Tr, <, e). Also, we can tell apart wellorders from non well founded relations. 
Define by recursion on a < oj\ the following modal formulas: 

• y?o = T. 

• ip a+1 = §(p a . 

9 ¥>x = A/3<aV 9 /3, for limit A. 

Proposition 12. For a wellorder (M, R) of type a, Tr, e \= (pp if and only if {3 < a. If 
(N, R) is not well founded, T R , e (= tpp for all /3 < U\. 



S h ...S n e T R 
tl, ■ ■ ■ t m f. Tr, 
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Corollary 13. If (N, R) is a wellorder and R! G LO, T R ,e ~ T R ,,e (N, R) 

(N,R'). 



It is well-known (J. Stern [13|) see that the class WO <a of wellorders of type less than 
a < Ui has arbitrary large Borel complexity with a big enough. We will use this fact and 
the previous lemmas to prove that bisimilarity is not Borel. But first we must merge the 
tree processes Tr into a single MLTS. 

Let F = (F,B{F),<), where: 

• (F, B(F)) = (Tr N x N*, £>(Tr N x N*)) (where N* is considered discrete). 

• <(T, s) = {(T, s') : s, s' G T & s < s'}. 

We prove that F is a MLTS. First note that sets <((T,s)), being countable, are Borel. 
We will also use the symbol < also as a binary relation, defined in the obvious way: 
(T, s) < (T", s') iff T = T', s, s' G T and s < s'. 

For subset A C X x K of a product and c G K the section A\ c is the set {x G X : 
(x, c) G A}, the preimage of the injection x i-> (x, c). 

Lemma 14. 0<5 Borel for each Q G £>(Tr N x N*). 

Proof. 

OQ = {(T, s) G F : 3(T', s') G Q ((T, s ) < (T f , s'))} 
= {(T,s)eF: 3s' ((T, s') eQ&(T,s)< (T, s'))} 
= {(T, s)eF:3ne N(s~n G T & (T, s~n) G Q)} 
= U U s)eF:s^neTL (T, G Q} 

neNsSN* 

Now we may write the set inside the unions (now for fixed s,n) as 

{(T,s) G F : G T & (T, s~n) GQ}=(Qfl {(T, s~n) : G T}) | s ~ n x {s} 

= (Q n ({T : A G T} x {s~n})) | s ~ n x {s} 

The inner rectangle is clopen, and since Q is Borel, the set between the big parentheses 
is Borel. The whole set is easily Borel, too. □ 

Theorem 15. ~ is not a Borel subset of F x F. 

Proof. It is clear that the injection T t— > (T, e) is continuous from Tr^ to Tr^ x N*; hence 
the composition R — )■ Tr ; = (Tr, e) also is (by Lemma ITTj) . If ~ were S°, then [Tr]~ = 
(~)|f„ would also be for every R. For R a wellorder of type a, WO a = (T.)~ 1 ([Tr]^) 
by Corollary [H This implies that WO <a = [j p<a WO <a would be 5^ for all a < u u 
contradicting the results in [13]. □ 

Lemma 16. If J*f is a countable logic that characterizes bisimulation and \S£\ C 13(F), 
then there exists a < ui 1 such that for all s, [s]„ is 11° (F). 
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Proof. Since ft([J&?]) = ~, 

w~ = n^M : ^ s i= n pk f \ m -f, (i) 

Take (3 = sup{7 < co x : [</?] £ S°,<^ G if}. Now, for all <p G if, [p] G Sj +1 , 
F\ G n° +1 , and countable intersections of such sets lie in n° +2 . Hence for a = (3 + 2, 
all sets in (|XJ) lie in 11°. □ 

By combining the proof of Theorem [15] and the previous Lemma we obtain 

Theorem 17. There is no countable logic if that characterizes bisimulation on F such 
thatlJfjCB(F). 

3.2 Bisimilarity is T,\ 

We finally show in this section that bisimilarity on F behaves similarly to the isomorphism 
relation: it is an analytic equivalence relation with Borel classes. 

We will need a technical tool from Janin and Walukiewicz [9|, adapted to our mono- 
modal case. An uj-indexed path from s G S on a LTS S = (S, R) is a sequence u of the 
form 

u = s (si, ai)(s 2 , a 2 ) . . . (s n , a n ) 

such that so = s, flj G N for all i, and (sj_i, si) G i? for z = 0, . . . , n. The u-expansion 
at s of a LTS S is the LTS f2g(s) = (fl, .R) such that f2 is the set of all u- indexed paths 
from s of S and the relation R is defined by (u, v) G R iff v has the form u(s, a) for some 
a and s. 

Since we are dealing with trees on N, the latter construction provides us with another 
tree that it is easily seen to be isomorphic to the one given by the following alternative 
description. 

Definition 18. The u-expansion of T = (T, <) at s is the LTS Ht(s) = (Ot(s), Rt(s)) 
such that Q T {s) = {(t,n) : s^teT&neu} and the relation R T {s) C (T x N) 2 is 
given by 

(u, n) Rt(s) (t, m) •<=>■ u<t. 
If T is understood from the context will just write fl(s) = (f2(s), R{s)). 

It can be proved that two states in a tree are bisimilar if and only if have isomorphic 
w-expansions. 

Lemma 19. Bisimilarity classes on F are Borel. 

Proof. By the previous observation we conclude that [(T, s)]~ = _R~ 1 ([i?((T, s))]^). By 



Scott's Theorem 12], we know that [R((T, s))]^ is a Borel subset of 2^ N * xN ) . Then we 



just have to show that the map R(-) : Tr N xNM 2( N * xN ) given by 
(u, n) R((T, s)) (t, m) sCuLu<tkteT 
is Borel measurable. 
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For a finite set P = {((uj, n,), {t^mA) : i = 1, . . . , k} of elements of (N* x N) 2 , we 
have 

R-\{R : P CR}) = {(T, s) : U G T & s C u { % = 1, . . . , k} 
= {T:Vi(t i eT)}x{s:Vi(sC Ui )}, 

if \/i : Ui< ti, and it is empty otherwise. 

This proves that the map is continuous and, a fortiori, Borel measurable. □ 

By using the previous reduction, one can prove that bisimilarity is Xj, since isomor- 
phism is. We also give a direct proof of this fact. 

Lemma 20. Let A be countable with the discrete topology, Y Polish and B k C Y Borel 
for all k G A. Then 

C{R,y) ^ VkeR:(yeB k ) 

is Borel in 2 A x Y . 

Proof. We have (R, y) G C VA; G A : (k G R y G B k ). Then 

C = f|{(i2,j/):A;ei2=^j/eB fc } 

= ^\{(R,y):kiR}U{(R,y):yeB k } 

keA 

= f)({R: k R} xY)U (2 A x {y : y e B k }) 

keA 

= p| ({R : {k} n i? = 0} x y) U (2 A x s fc ) 

which is obviously Borel. □ 
Theorem 21. Bisimilarity onF is H{. 

Proof. As usual, n, m denote non negative integers and Sj finite sequences. The definition 
of bisimilarity on F is as follows: 

(Tx, s) ~ (T 2 , s') 3R G 2™* : (s, s') G i? & 

k VsiVs 2 Vn. (sfn G 7\ & s 2 G T 2 & {s u s 2 ) e R 

3m : s 2 ~m G T 2 & (si^n, s^m) G i? j & 

& VsiVs 2 Vn. ^Si G Ti & s 2 ^n G T 2 & (s l5 s 2 ) G i? =>• 

3m : si^m G Xi & (si~m, s 2 "n) G i?j . 

It suffices to prove that the set defined inside the outer existential quantifier is Borel in 
2 N * xN * x F x F. We first consider the third line of the definition. The set defined by 

(R, (7\, s), (T 2 , s')) G ^ (si)S2 ),n,m <^=> s 2 ^m G T 2 & (sfn, s 2 ~m) G i? 



is easily Borel. Then the condition 3m : s 2 "m G T 2 & (si^n, s 2 ^m) E -R also is and 

Vra. (^si^n 6 Ti & S2 G T 2 =>- 3m : s^m E T 2 & (si~n, s 2 "m) £ -R^ 

finally defines a Borel set of tuples (R, (Xi, s), (T 2 , s')) indexed by elements (s\, s%) E R. 
We may apply now Lemma 1201 and conclude that 

V(si, s 2 ) E R : Vn. ^si^n G Ti & s 2 G T 2 => 3m : s 2 "m E T 2 & (si~n, s 2 "m) G i?j , 

which is equivalent to the second and third lines of our definition for bisimilarity, is Borel. 
The rest of the formula is handled similarly. □ 

4 Conclusion 

In the framework of nondeterministic labelled Markov process, we considered the problem 
of describing bisimilarity by using a modal logic. We studied this problem in a very 
restricted case, that of image-countable processes where the only kind of probabilities 
considered are Dirac measures. 

In this restricted setting, all proposed definitions of bisimilarity coincide (in particular 
with the standard bisimilarity notion for labelled transition systems). We proved that the 
relation of bisimilarity is a analytic non Borel subset of F 2 , where F is the base space of 
a model consisting of trees, and hence there is no countable Borel logic that characterizes 
bisimilarity for this process. 

Indeed, Hennessy-Milner logic already characterizes bisimilarity in this case, and it 
is a Borel logic (obviously with uncountable many formulas). But in order to cope with 
image-countable NLMP with no restrictions on the kind of probabilities we use, this 
is a serious limitation: to our knowledge, all the proofs for logical characterization of 
bisimilarities are based on the structure of analytic spaces, and evidence shows that one 
needs a countable Borel logic to be able to use this structural properties. 
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